fstats_missing_data Module

Provides routines for dealing with data sets containing missing observations. Missing values are denoted by IEEE quiet NaN's.

Three imputation strategies are provided.

  • k-nearest neighbors imputation (knn_impute): a nonparametric, donor-based approach.

  • Likelihood-based imputation (em_impute): maximum-likelihood estimation of the mean vector and covariance matrix of a multivariate normal population by means of the expectation-maximization (EM) algorithm. The missing values are replaced by their conditional expectations.

  • Multiple imputation (multiple_impute): Bayesian data augmentation under a multivariate normal model that generates several completed copies of the data set such that the uncertainty introduced by the imputation process can be propagated through subsequent analyses. The results of the analyses of each completed data set can be combined by means of Rubin's rules (pool_imputations).



Functions

public elemental function is_missing(x) result(rst)

Determines if the supplied value represents a missing observation.

Arguments

Type IntentOptional Attributes Name
real(kind=real64), intent(in) :: x

The value to test.

Return Value logical

Returns true if the value is missing; else, false.

public pure function missing_value() result(rst)

Returns the value used to flag a missing observation. The value is an IEEE quiet NaN.

Arguments

None

Return Value real(kind=real64)

The missing-value flag.


Subroutines

public subroutine em_impute(x, xc, mu, sigma, maxiter, tol, niter)

Employs the expectation-maximization (EM) algorithm to compute the maximum-likelihood estimates of the mean vector and covariance matrix of a multivariate normal population from a data set containing missing values. The missing values are then replaced by their conditional expectations given the observed values and the converged parameter estimates.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=real64), intent(in), dimension(:,:) :: x

An N-by-M matrix containing N observations of M variables. Missing entries must be denoted by NaN's (see missing_value).

real(kind=real64), intent(out), dimension(:,:) :: xc

An N-by-M matrix where the completed data set will be written.

real(kind=real64), intent(out), optional, dimension(:) :: mu

An optional M-element array where the maximum-likelihood estimate of the mean vector will be written.

real(kind=real64), intent(out), optional, dimension(:,:) :: sigma

An optional M-by-M matrix where the maximum-likelihood estimate of the covariance matrix will be written.

integer(kind=int32), intent(in), optional :: maxiter

An optional input specifying the maximum number of iterations to allow. The default is 500.

real(kind=real64), intent(in), optional :: tol

An optional input specifying the convergence tolerance. The iteration terminates once the largest change in any parameter falls below this value. The default is 1.0e-8.

integer(kind=int32), intent(out), optional :: niter

An optional output containing the number of iterations performed.

public subroutine knn_impute(x, xc, k, weighted)

Imputes missing values by means of a k-nearest neighbors approach.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=real64), intent(in), dimension(:,:) :: x

An N-by-M matrix containing N observations of M variables. Missing entries must be denoted by NaN's (see missing_value).

real(kind=real64), intent(out), dimension(:,:) :: xc

An N-by-M matrix where the completed data set will be written.

integer(kind=int32), intent(in), optional :: k

An optional input specifying the number of neighbors to use. The default is 5.

logical, intent(in), optional :: weighted

An optional input that, if set to true, weights each donor by the inverse of its distance. The default is true.

public subroutine multiple_impute(x, xc, nburn, maxiter, tol)

Generates multiple completed copies of a data set containing missing values by means of Bayesian data augmentation under a multivariate normal model with the noninformative Jeffreys prior.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=real64), intent(in), dimension(:,:) :: x

An N-by-M matrix containing N observations of M variables. Missing entries must be denoted by NaN's (see missing_value).

real(kind=real64), intent(out), dimension(:,:,:) :: xc

An N-by-M-by-K array where the K completed data sets will be written.

integer(kind=int32), intent(in), optional :: nburn

An optional input specifying the number of data augmentation cycles to perform between successive imputations. The default is 20.

integer(kind=int32), intent(in), optional :: maxiter

An optional input specifying the maximum number of EM iterations to allow when computing the starting estimates. The default is 500.

real(kind=real64), intent(in), optional :: tol

An optional input specifying the convergence tolerance used when computing the EM starting estimates. The default is 1.0e-8.

public pure subroutine pool_imputations(est, vars, qbar, tvar, dof)

Combines the results of the analyses of multiply-imputed data sets by means of Rubin's rules.

Arguments

Type IntentOptional Attributes Name
real(kind=real64), intent(in), dimension(:) :: est

A K-element array containing the parameter estimate obtained from each of the K imputed data sets.

real(kind=real64), intent(in), dimension(:) :: vars

A K-element array containing the estimated variance of the parameter obtained from each of the K imputed data sets.

real(kind=real64), intent(out) :: qbar

The pooled parameter estimate.

real(kind=real64), intent(out) :: tvar

The total variance of the pooled estimate. The total variance is the sum of the within-imputation variance and the appropriately scaled between-imputation variance.

real(kind=real64), intent(out), optional :: dof

An optional output containing the degrees of freedom associated with the pooled estimate.